In this paper, we propose to use the scatter of celestial pole offset (CPO) series obtained from VLBI observations as a measure of the accuracy of the celestial reference frame (CRF) realizations. Several scatter indices (SIs) including those proposed for the first time are investigated. The first SI is based on analysis of residuals of CPO series w.r.t. Free Core Nutation (FCN) model. The second group of SIs includes Allan deviation and its extensions, which allow one to treat unequal and multidimensional observations. Application of these criteria to several radio source catalogues showed their ability to perform a preliminary assessment of the quality of radio source catalogues, and 2D Allan deviation estimate seems to be a most sensitive measure. However, in common case, the sensitivity of tested criteria is yet not sufficient to distinguish between radio source catalogues of the highest quality. Proposed extensions of Allan deviation, weighted and multidimensional, can be effectively used also for statistical analysis of astro-geodetic and other time series.